A function is α-strongly convex and β-smooth if for all 𝐱,𝐲\mathbf{x},\mathbf{y}:

α2||𝐲−𝐱||22≤[f(𝐲)−f(𝐱)]−∇f(𝐱)𝖳(𝐲−𝐱)≤β2||𝐲−𝐱||22\frac{\alpha}{2}||\mathbf{y}-\mathbf{x}||_2^2 \leq [f(\mathbf{y})-f(\mathbf{x})]-\nabla f(\mathbf{x})^\mathsf{T}(\mathbf{y}-\mathbf{x}) \leq \frac{\beta}{2}||\mathbf{y}-\mathbf{x}||_2^2

(multidimensional generalization)

For scalar functions, a twice-differentiable function ff is α-strongly convex and β-smooth if for all xx,

α≤f′′(x)≤β\alpha \leq f''(x) \leq \beta

If f is β-smooth and α-strongly convex then at any point 𝐱\mathbf{x}, the Hessian ∇2f(𝐱)∇^2 f(\mathbf{x}) satisfies:

α𝐈⪯∇2f(𝐱)⪯β𝐈\alpha \mathbf{I} \preceq ∇^2 f(\mathbf{x}) \preceq \beta \mathbf{I}

where 𝐈\mathbf{I} is a d×dd \times d identity matrix.

This is the natural matrix generalization of the statement for scalar valued functions.

Note the PSD relations

Equivalently for any 𝐳\mathbf{z},

#incomplete

condition number

κ=βα\kappa = \frac{\beta}{\alpha} is called the condition number of ff